


»\ Fy The ° ADVANCED 
A college AP riacemen: 
Board PROGRAM’ 


AP® Statistics 
1999 Scoring Guidelines 











The materials included in these files are intended for non-commercial use by AP 
teachers for course and exam preparation; permission for any other use must be 
sought from the Advanced Placement Program. Teachers may reproduce them, in 
whole or in part, in limited quantities, for face-to-face teaching purposes but may 


not mass distribute the materials, electronically or otherwise. These materials and 
any copies made of them may not be resold, and the copyright notices must be 
retained as they appear here. This permission does not apply to any third-party 
copyrights contained herein. 





These materials were produced by Educational Testing Service (ETS), which develops and administers the examinations of the Advanced Placement Program for the 
College Board. The College Board and Educational Testing Service (ETS) are dedicated to the principle of equal opportunity, and their programs, services, and 
employment policies are guided by that principle. 


The College Board is a national nonprofit membership association dedicated to preparing, inspiring, and connecting students to college and opportunity. 
Founded in 1900, the association is composed of more than 3,900 schools, colleges, universities, and other educational organizations. Each year, the College Board 
serves over three million students and their parents, 22,000 high schools, and 3,500 colleges, through major programs and services in college admission, guidance, 

assessment, financial aid, enrollment, and teaching and learning. Among its best-known programs are the SAT", the PSAT/NMSQT™, the Advanced Placement 
Program” (AP®), and Pacesetter”. The College Board is committed to the principles of equity and excellence, and that commitment is embodied in all of its 
programs, services, activities, and concerns. 


Copyright © 2001 by College Entrance Examination Board. All rights reserved. College Board, Advanced Placement Program, AP, and the acorn logo are registered 
trademarks of the College Entrance Examination Board. 


AP* STATISTICS 
1999 SCORING GUIDELINES 


Question 1 


Solution: 


Notes: 


Yes. Test for slope indicates that the linear model is useful (H,: BETA is equal to 0, H,: 
BETA is not equal to 0, t = 54.11, p-value = .000) and the residual plot shows no pattern, 
indicating a linear model is appropriate. 


Slope = 233.517 aircraft/year 
On average, the number of commercial aircraft flying in the U.S. increased by 
approximately 233.517 each year. (OK if rounded to 234 in interpretation) 


Intercept = 2939.93 aircraft 
Predicted number of commercial aircraft that were flying in 1990 (since x = 0 
corresponds to year 1990) was 2939.93. (OK if rounded to 2940 in interpretation) 


For 1992, x = 2, so predicted number of commercial aircraft flying is 
2939.93 + 233.517(2) = 3406.964 aircraft 


From the residual plot, the residual for 1992 is +40, so actual - predicted = 40 and 
Actual = 3406.964 + 40 = 3446.964 aircraft 
Since actual number flying must be an integer, actual must have been 3447. 


Part (a) can be considered essentially correct even if it fails to mention the t test, as long 
as it discusses the residual plot. 


Parts (b) and (c) should draw the distinction between the model and the data. They can be 
considered essentially correct if the student incorporates the idea of estimation using 
words such as on average, predicted, approximately, about, etc. 


Parts (b) and (c) can be considered partially correct if the student (1) incorrectly identifies 
the values for the slope and intercept but gives an essentially correct interpretation OR (2) 
correctly identifies the values for the slope and intercept but gives an incomplete 
interpretation or an interpretation not in context for one or both. 


Parts (d) and (e) can be considered essentially correct if incorrect numbers from previous 
parts are correctly substituted. 


Part (e) can be considered essentially correct even if it fails to round to an integer. 
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Question 1 (cont.) 
Points: 


4 Complete Response 
Gives an essentially correct response to all 5 parts. 


3 Substantial Response 
Essentially correct on 4 of the 5 parts. 
OR 
Essentially correct responses on a, d, and e 
AND 
partially correct responses on both b and c. 


2 Developing Response 
Essentially correct on 3 of the 5 parts. 
OR 
Partially correct responses on both b and c 
AND 
essentially correct responses on 2 of the remaining parts 


1 Minimal Response 
Essentially correct on 1 or 2 of the 5 parts. 
OR 
Partially correct responses on both b and c 
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Question 2 


Solution: 
H,: There is no association between level of hiking experience and direction 


H,: There is an association between level of hiking experience and direction 
or 

H,: Level of hiking experience and direction are independent 

H,: Level of hiking experience and direction are not independent 


Chi-Square test for independence 
2 
2 vl Obs— Exp) 
, a — 
Exp 
Requirements: Need to check expected counts using some accepted rule (textbooks differ). 
Table of Observed and (Expected Counts): 


20 (18)|50 (48)/50 (54) 
10 (12)|30 (32)|40 (36) 


Requirements Check: All expected counts are greater than or equal to 5. So the Chi-Square test 
of independence is appropriate. 


(20-18) (50-48) (50-54 (10-12) (0-32) (40-36) 
a ‘5 4 + i: es = 1.5046 
18 48 54 12 32 36 








df = 2, P-value = 0.471 OR (using tables) P-value > 0.25 

Since the P-value is large for any reasonable level of significance, we fail to reject H,. (A picture 
showing an appropriate rejection region and the test statistic value is acceptable). There is not 
convincing evidence that an association exists between level of hiking expertise and direction. 


Note: 

For rejection region approach, rejection regions are 
x° > 9.21 or significance level 0.01, 

x’ > 5.99 for significance level 0.05, 

x’ > 4.61 for significance level 0.10 
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Question 2 (cont.) 
A complete response constitutes 


Stating a correct set of hypotheses, 

Identifying appropriate test & checking appropriate assumptions, 

Providing correct mechanics, and 

Stating correct conclusion in the context of the problem using results of the statistical test. 


aos 


Part (a) Stating a correct set of hypotheses, 


e States null and alternative hypothesis correctly defining any notations used. 
e Both hypotheses are stated in the context of problem. 


Note: If a student switches the null and alternative hypotheses, then no credit will be given for 
part (a). 


Part (b) Identifying appropriate test & checking appropriate requirements, 


e Gives some indication of using a correct statistical test by naming the test, by giving correct 
symbol or formula for the test statistic. 

e Checks requirements. In this case, expected counts are greater than or equal to 5. Some 
indication of use of expected counts in checking requirements is needed. 


Note: If a wrong statistical test is performed, then no credit is given for section (b). 
Part (c) Providing correct mechanics, 


e Gives the computed test statistic value correctly. 
e Gives the associated P-value correctly (the rejection region approach is acceptable) 


Note: One minor computational error will not necessarily drop the score to a 3. 


Part (d) Stating correct conclusion in the context of the problem using results of the statistical 
test. 


e Uses test results correctly to arrive at the conclusion (either p-value approach or the rejection 
region approach). Linkage between the conclusion and the test result must be stated clearly. 
Correct interpretation of a P-value is acceptable as a statistical linkage. 

e Writes conclusion in the context of the problem consistent with the defined hypotheses. 


Note: If a student applies rejection rule incorrectly, then no credit will be given for part (d). 
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Question 2 (cont.) 


Points: 
4 Complete Response 
Essentially correct responses on all 4 parts. 


3 Substantial Response 
Essentially correct responses on any 3 of the 4 parts. 
OR 
Essentially correct responses on any 2 of the 4 parts and partially correct responses on the 
remaining 2 parts. 


2 Developing Response 
Essentially correct responses on any 2 of the 4 parts. 
OR 
Essentially correct response on any | of the 4 parts and partially correct responses on any 2 
of the remaining 3 responses. 
OR 
Partially correct responses on all 4 parts. 


1 Minimal Response 
Essentially correct response on 1 of the 4 parts. 
OR 


Partially correct responses on 2 or 3 of the 4 parts. 
Notes: 


1. Maximum score of 2 if the student does a Chi-square on the proportions rather than the 
counts. 

2. Some students may attempt to do the problem using two-proportion tests. Bring these 
solutions to the table leaders. 

3. Score is 0 if there is no statistical test used (e.g. conclusion based on the comparison of 
proportions only). 
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Question 3 
Note To Readers: 


Holistic grading is essential for evaluating student responses to this question. In judging the 
responses for each individual question as correct, weak, or incorrect, the total response to all 
questions (a), (b), and (c) should be read and evaluated. If the reader can construct an 
unambiguous correct response to (a), for example, from the student's total response, then 
question (a) should be counted as "correct." 


Solution 
This problem has three parts. 


a. The student can appeal to any of three reasons in judging this study not an experiment: 


1. there is no random assignment of subjects to treatments; 
2. there are no treatments imposed; 
3. existing data is being used. 


b. Two variables are confounded if their effect on the number of new cavities cannot be 
distinguished from one another. The student must mention not only that the confounding 
variables may affect the outcome but that they have differential effects within the two 
groups. For instance: confounding would occur if patients who eat an apple a day differ from 
those who eat less than one apple a week on some variable that is related to dental health. 
In this example, diet or general level of health are examples of what might be confounding 
variables. For example, it is possible that people who eat an apple a day are more nutrition 
conscious and have a more healthy diet in general than those who eat one or fewer apples per 
week, and this might explain the observed difference in dental health. 


Note: 

There are many possible examples of confounding variables. Any reasonable example of a 
confounding variable is acceptable, as long as a good explanation is given and the connection 
between the confounding variable and group membership is clear. Lack of a definition here can 
be rectified by a response in (c) that demonstrates a clear understanding of the concept of 
confounding variable. 


c. No, because it is not an experiment, and cause-and-effect conclusions cannot be drawn from 
an observational study. 


OR 


No, because there are possible confounding variables. 
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Question 3 (cont.) 


Notes: 


In b), a good definition of confounding with a bad example should be regarded as 
temporarily weak. An example that does not mention group affiliation can be recovered in 
part c). To recover the definition of confounding in c) they must connect the term with the 
definition. To recover the group affiliation, they may do by example in c). 


If the student, in attempting to discuss group differentiation, only mentions one of the groups, 
that is OK - we will consider the other implied. For example, it is counted correct if the 
student says, "The apple-eating group may be more health-conscious..." they need not 
explicitly deny health-consciousness to the one-apple-a-week group. 


The constructions "Some people" and "A person in the apple-eating group may ..." are NOT 
enough to establish group differentiation; this construction suggests only that some subset of 
the group may differ from the rest of the group. This is just natural variation. The 
construction "A person who is an apple-eater may ..." can establish group differentiation if it 
is clear that this is describing a representative member of the group. 


Mentioning initial non-equivalence of groups without tying that non-equivalence to the 
outcome is not correct. Mentioning concepts such as self-reporting bias, social desirability, 
etc. may constitute measurement error in the study but is not confounding. 


In (c), appealing to the definition of confounding variables in (b) would get a "correct" for (c) 
if the definition in (b) is correct. If the definition in (b) is weak, that appeal alone would not 
get credit. 


If the definition in (b) is "there are other variables that affect the outcome measure, such as 
age, health, etc," this is not regarded as a correct definition for purposes of appealing to the 
definition from (c). For purposes of reading part (b), this definition would be regarded as 
weak. 


If in (c), they give an example which is the equivalent of confounding, and refer to this as 
confounding, they would get credit for (b). 
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Question 3 (cont.) 
Points: 


4 Complete Response 
All 3 of the following must be correct: 
e Gives a correct explanation of why this is an observational study or is not an experiment. 
e Gives an acceptable explanation of confounding and a reasonable example of a possible 
confounding variable 
e Correctly states that a causal relationship cannot be inferred from the results of an 
observational study 
e 
3 Substantial Response 
2 correct and | weak response. 


2 Developing Response 
2 correct and | incorrect 
OR 
1 correct and 2 weak 


1 Minimal Response 
1 correct 
OR 
2 or 3 weak 


0 1 weak 


Copyright © 2001 by College Entrance Examination Board. All rights reserved. 
Advanced Placement Program and AP are registered trademarks of the College Entrance Examination Board. 


9 


AP* STATISTICS 
1999 SCORING GUIDELINES 


Question 4 
Solution 
1. Plan A: defective if more than two standard deviations below the mean 


For Plan A 
P(defective) = P(z < -2) = .0228 (Tables) (.02275 Calculator) 


So, plan A rejects .0228 or 2.28% as defective. 
OR 


Students use the Empirical Rule: about 95% of the area under the normal curve lies between z = - 
2 and z = 2, so 5% lies outside these limits, and about 2.5% lies below z = -2. 


2. For Plan A: 


P(at least 1 of 2 rejected) 
= | - P(none rejected) 
= 1-2 nCr 0 (0.0228)° (0.9772) 
=1-(.9772) 

1 - .9549 

= .0451 (.04498 Calculator) 


OR 


P(at least 1 of 2 rejected) 
= P(exactly 1 rejected) + P(exactly 2 rejected) 
=2 nCr 1 (0.0228) (0.9772) + 2 nCr 2 (0.0228)° (0.9772)° 
2 (.0228) (.9772) + (0.0228)° 
.0446 + .0005 
= .0451 


OR 


P(at least 1 of 2 rejected) 
= P(\st rejected) + P(2nd rejected) - P(both rejected) 
= 0228 + .0228 - (.0228) 

0451 


II 


Il 


II 


II 


Note: If P(defective) = .025 then P(at least 1 of 2 rejected) = .04938 
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Question 4 (cont.) 


3. Plan B: defective if more than 1.5 IQR below the lower quartile 
Quartiles are at .67 std. dev. (.67449 Calculator) 
IQR = 1.34 std. dev. (1.34898 Calculator) 
1.5 IQR = 2.01 std. dev. (2.02347 Calculator) 
Boundary for defective region is: 
Q1 - 1.5 IQR = - .67 - 2.01 = - 2.68 
For Plan B 
P(defective) = P(z < -2.68) = .0037  (.00349 Calculator) 
So, Plan B rejects .0037 or .37% as defective. (Tables) (.349% Calculator) 


Points: 


4 Complete Response 
Except for minor arithmetic errors 


e Correctly computes the defective proportion in (a) and shows appropriate work 

e Correctly computes the probability in (b) and shows appropriate work 

e Correctly determines boundary for defective region in part (c) and the corresponding 
defective proportion 


3 Substantial Response 
Except for minor errors 


e Correctly computes the defective proportion in (a) and shows appropriate work 
Correctly computes the probability in (b) and shows appropriate work 
Makes an error in determining the IQR or boundary for the defective region in (c) but 
computes the proportion defective correctly for the stated boundaries 


A paper can also get a 3 if the defective region is seen as two sided (i.e. more than two std. dev. 
from the mean rather than more than two std. dev. below the mean) and answers to all three parts 
are correct for this interpretation. 
(a: .0456, b: .0891, c: .0074 ; or on Calculator: a: .0455, b: .0889, c: .006976) 

OR 
Part (c) is essentially correct and either (a) or (b) is correct 
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Question 4 (cont.) 


2 Developing Response 
Answers only parts (a) and (b) correctly (for the one- or two-sided interpretation) 
OR 
Answers part (a) incorrectly, but uses a correct method in (b) given the student's computed 
answer from (a), and part (c) is partially correct. 
OR 
Answers part (c) essentially correctly, but parts (a) and (b) are incorrect. 


1 Minimal Response 
Answers part (a) essentially correctly (for the one- or two-sided interpretation) but parts (b) and 
(c) are incorrect or blank 

OR 
Answers parts (a) and (c) incorrectly, but uses a correct method in (b) given the student’ 
computed answer from (a) 

OR 
Answers part (c) partially correct, but parts (a) and (b) are incorrect or blank 
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Question 5 
Solution 
Possible Outcomes 
Die A Die B Winner Prob 

9 3 A (2/3)(2/3) = 4/9 

9 11 B (2/3)(1/3) = 2/9 

0 3 B (1/3)(2/3) = 2/9 

0 11 B (1/3)(1/3) = 1/9 
OR 

DIEA 








Winner Prob 
A 16/36 = 4/9 
B 20/36 = 5/9 


a. Choose die B, because the probability of winning is higher (5/9 compared to 4/9 for die A) 
b. Let X be the number of tokens the player using die B should receive. For the game to be fair, 
we need 


45(4/9) = X(5/9) 
Solving this equation for X gives X = 36. Player B should receive 36 tokens. 
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Question 5 (cont.) 
Points 


4 Complete Response 
Part (a) and Part (b) must both be correct. 


3 Substantial Response 
Part (a) or Part (b) is correct and the other part is partially correct. 


2 Developing Response 

Part (a) or Part (b) is correct and the other part is incorrect. 
OR 

Part (a) and Part (b) are both partially correct. 


1 Minimal Response 
Part (a) or Part (b) is partially correct. 


Correct Responses 
Part (a) 


e Correctly reasons*, using a probability argument, that die B should be chosen. 
Part (b) 


e Gives a correct answer* to part (b) and gives a complete explanation of how the answer was 
determined. 


Partially Correct Responses 
Part (a) 


e Chooses the correct die, has a correct approach, but the explanation is incomplete. 
e Gives an incorrect answer to part (a) based on a correct approach. 


e Chooses the incorrect die, A, based on an expected value argument, but correctly computes 
expected values of 6 for die A and 5.667 for die B. 


Part (b) 


e Arrives at an answer, has a correct approach, but with an incomplete explanation. (A 
common example of this is not demonstrating the use of probabilities.) 
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Question 5 (cont.) 


Incorrect Responses 
Part (a) 


e Uses an expected value argument but gets the wrong expected values. 

e Choosing die A because the probability of rolling a 9 on Die A = 2/3 and the probability of 
rolling an 11 on die B= 1/3. 

e An answer without an explanation receives no credit. 


Part (b) 
e An answer without an explanation receives no credit. 


Notes: 

e Papers with minor errors, such as transposition errors or arithmetic errors, that give 
reasonable answers, should not be penalized. 

e Another possibility is that the student takes a simulation approach to the problem. It is 
possible for this approach to earn up to a score of 4. Papers taking this approach should be 
referred to a table leader. 
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Question 6 
Points: 


Each part (a), (b), and (c) is essentially correct, partially correct, or incorrect. 


4 Complete Response 
Essentially correct on all 3 parts 


3 Substantial Response 
Essentially correct on 2 parts, 
OR 
Essentially correct on part (a) or (b) and partially correct on other 2 parts 


2 Developing Response 

Essentially correct on part (c) and partially correct on either part (a) or (b) 
OR 

Partially correct on at least 2 parts 
OR 

Essentially correct on part (a) or (b) 


1 Minimal Response 
Essentially correct on part (c), but incorrect on the other two parts. 
OR 
Partially correct on any | part 
OR 
Holistically partially correct overall (e.g. recognizes need for matched pairs in part (b)) 


Key Elements in Solutions: 
a. This question focuses on just the score after training to assess whether performance is 
better than 50%. 
This question focuses on assessing change. 
c. This question focuses on correlation or regression slope to assess association. 


Complete Solutions 


a. Let MU, = mean number of correct responses after training 


Ho: MU, = 50 (or <=) Ha: MU, > 50 


One-sample t test Test statistic: o> 
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Question 6 (cont.) 
Assumption: Normal population distribution. 


Checking Assumption for t test: Boxplot, dot plot, stem and leaf plot or histogram does not show 
any outliers or extreme skewness. Or normal probability plot looks OK. (See attached plots.) 


X= 56.3 
Sa = 4.725 
n= 20 
ze §6.3-—50 
A725 /4j20 _ ee 
df= 19 


P-value = .0000049 


For any reasonable choice of ALPHA, reject H,. There is convincing evidence that the mean 
number of correct responses after training is higher than 50, the value expected by chance alone. 


(For a rejection region approach, Rejection Region boundaries are at 1.33 for ALPHA = .1, 1.73 
for ALPHA = .05, and 2.54 for ALPHA= .01) 


Other possible approaches scored as Essentially correct: 


e Sign test (1 below 50, 2 at 50, and 17 above 50. P-value = .0001). Doesn’t require 
normality assumption. 

e Binomial test or Z test based on the normal approximation to binomial (H,: p = .5, H,: p > 
.5, assumptions: large sample (e.g. np,.>=5, n(1-p,)>=55), ok to combine across 
individuals, n = 2000, = 1126/2000 = .563, z = 5.63, P-value = .000000008782). 

e A chi-square test can be a correct method (x’=48.72, df=20) but has a minor error in that 
it assumes a two-sided alternative. 


b. The analysis in part (a) does not provide evidence that the training is effective, because it 
does not compare before and after scores. Data is paired. 
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Question 6 (cont.) 


H,: MU, = 0 Where MUD is the mean difference in the score before training and the score after 
training 


H,: MU, > 0 (would be < if student defines differences as before-after) 
OR 


H,: MUz - MU, = 0 H,: MU - MU, > 0 ( < if student defines differences as before-after) 


One-sample t test — Test statistic: ae _ 


Checking Assumption for t test: Boxplot, dot plot, stem and leaf plot or histogram of sample 
differences does not show any outliers or extreme skewness. Or normal probability plot of 
differences looks OK. (See attached graphs.) 


Assumption: Normal difference population 


x= 3 
Sp = 6.837 
n= 20 
3-0 
t = ———_— = .1962 
6.837 / 420 
df= 19 


P-value = .4233 


For any reasonable choice of ALPHA, fail to reject Ho. There is not convincing evidence that 
training is effective in improving a person’s ability to predict coin tossing outcomes. 


(For a rejection region approach, Rejection Region boundaries are at 1.33 for ALPHA = .1, 1.73 
for ALPHA= .05, and 2.54 for ALPHA = .01) 


An alternate correct approach is to use the sign test (11 negative differences, 1 zero difference, 
8 positive differences, P-value = .6762). 
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Question 6 (cont.) 


Alternate incorrect approaches include using a two independent sample t test (t = .1943, P- 
value = .4235 for either pooled or unpooled. ff = 38 for pooled, df = 37 for unpooled) or a two 
independent sample z test for proportions (z = .1912, p-value = .4242). (See definition of 
partially correct.) 


c. The response indicates that knowledge of a person's score before training would not be 
helpful in predicting after training score and justifies this response in at least one of the 
following ways: 


e A scatter plot of the after scores vs. the before scores shows no pattern indicating 
a relationship between before and after scores 

e The correlation coefficient for before and after scores is very close to 0 (r = .020 
or R’ = .0396%) 

e The test for slope in a regression of y = "after score" on x = "before score" does 
not indicate that the slope is significantly different from 0 (t = .08, P-value = 
934). 


65 


PYUV 


15 


45 
45 55 65 
Before 


A complete solution in (a) and (b) contains the following 5 steps 


Hypotheses are correct. 

Assumptions are stated and checked. 

The correct test is used and is either named or formula for the test statistic is given. 
Test statistic and p-value computed correctly. 

A correct conclusion is given in context. 
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Question 6 (cont.) 
Parts (a) and (b) are essentially correct if student correctly completes 4 or 5 of the above steps. 
Parts (a) and (b) are partially correct if student correctly completes 3 of the above steps. 


Parts (a) and (b) are incorrect if only 1 or 2 steps are completely correct. 
UNLESS: 


e Part (a) is considered Incorrect if the student uses both columns of data in the analysis 
(e.g. matched pairs, two independent samples, chi-square). 

e In part (b), a two using a two independent sample procedure (means or proportions) 
receives at most Partial credit in part (b). Student must get 3 of remaining 4 steps to 
receive the Partial credit in (b). 

e In parts (a) or (b), if the students manufacture an invalid formula (e.g. mixes t and 
binomial numerator and denominator) they receive at most Partial credit. 


If an error similar to the following is made, student does not receive credit for that step. 


e Inconsistent or undefined nonstandard notation. 

e Incorrect hypotheses (but if H, is 2-sided, don't count both this & 2-sided p-values as 
errors). 

e States but doesn't check assumptions. 

Plugs wrong numbers into correct formula (e.g. forgetting , uses n = 100, uses n = 20 in 

one-sample z-test for proportions). 

Uses z instead of t when SIGMA is unknown. 

Calculates two-sided p-value or one-sided in the wrong direction. 

Uses a two-sided confidence interval approach. 

Gives incorrect conclusion or conclusion not stated in context of problem. 


Part (c) is essentially correct if student examines the association between before and after 
scores, instead of the differences. 


Part (c) is partially correct if student merely recognizes the need to examine the association 
(e.g. say no pattern but gives no evidence has examined the plot or correlation coefficient). 
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Question 6 (cont.) 
Part (c) is incorrect if 


= Student only refers to results of (a) or (b): 
= Response indicates that since there was no significant difference between before 
and after, the before score is a useful predictor of the after score (should be about 
the same.) 
= Response indicates that since there was no significant difference, training doesn't 
matter, so the before score is not a useful predictor. 
* Response is based on the coin flips being fair and independent, rather than on the guesses 
of the outcomes. 
= Response examines association through a chi-square test using data given as two-way 
table. 
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